Nvidia's Blackwell chips are about twice as big as its predecessors, housing 2.6 times the number of transistors. Instead of one big piece of silicon, Blackwell consists of two advanced processors and numerous memory components joined in a single, delicate mesh of silicon, metal, and plastic. The manufacturing of each chip has to be close to perfect, presenting engineering challenges that have a sizable impact on the bottom line, with each defect rendering a $40,000 chip useless. This article looks at some of the challenges Nvidia had to overcome to produce the chip.

Nvidia's Blackwell B200 chips will take at least three months longer to produce than was planned. The delay is due to a design flaw that was discovered unusually late in the production process. Nvidia is now working through a fresh set of test runs and won't ship large numbers of the chips until the first quarter. Microsoft, Google, and Meta have already ordered tens of billions of dollars worth of the chips.

Nvidia has unveiled a new generation of artificial intelligence chip architecture called Rubin. The company only just announced its upcoming Blackwell model in March - those chips are still in production and expected to ship to customers later in 2024. Nvidia has pledged to release new AI chip models on a one-year rhythm. The less-than-three-month turnaround from Blackwell to Rubin underscores the competitive frenzy in the AI chip market.

Nvidia's new Blackwell chip demonstrated top per GPU performance in MLPerf's LLM Q&A benchmark, showcasing significant advancements with its 4-bit floating-point precision. However, competitors like Untether AI and AMD also showed promising results, particularly in energy efficiency. Untether AI's speedAI240 chip, for instance, excelled in the edge-closed category, highlighting diverse strengths across new AI inference hardware.

Nvidia is reportedly preparing a system-on-chip that pairs Arm's Cortex-X5 core design with GPUs based on Nvidia's Blackwell architecture.

Rumors suggest NVIDIA may introduce a new TITAN AI graphics card based on the Blackwell GPU. Tech leakers hint at this top-tier card's existence, despite NVIDIA's previous decision to not release a Titan variant for the RTX 40 series. The release and actual utility of such a high-performance GPU, potentially 63% faster than the RTX 4090, remain uncertain. Market dominance by the RTX 4090 may make a new Titan superfluous.

Nvidia is developing a new AI chip, the B20, tailored to comply with U.S. export controls for the Chinese market, leveraging its partnership with distributor Inspur. Its advanced H20 chip has reportedly seen a rapid growth in sales in China, with projections of selling over 1 million units worth $12 billion this year. U.S. pressure on semiconductor exports continues, with possible further restrictions and control measures on AI model development.

Nvidia's CEO Jensen Huang attributes the company's AI chip market dominance, maintaining an over 80% market share despite rising competition, to a decade-old strategic investment. Advocating for Nvidia's AI chips' cost-effectiveness and performance, Huang highlights the firm's transformation into a data center-focused entity and expansion into new markets.

Nvidia CEO Jensen Huang is trying to build Nvidia into a one-stop shop for all of the key elements in a data center. The strategy is designed to make the company's offerings stickier for customers. Nvidia is also building a business that supplies AI-optimized Ethernet, a business that is expected to generate billions of dollars in revenue within a year. The competition in the space is growing, with companies like AMD bolstering their data-center offerings and chip suppliers like Intel offering services and systems to help customers build and operate AI tools.

Nvidia's dominance in AI chips has propelled it to immense market value, largely thanks to its GPU capabilities and CUDA software ecosystem. However, competitors like AMD, Intel, Cerebras, and SambaNova are developing innovative solutions to challenge Nvidia's supremacy in AI hardware. While Nvidia's lead remains secure for now, the landscape is dynamic, with multiple players striving to carve out their own niches in the AI market.

Nvidia is facing increased government scrutiny from the EU, UK, China, and the US Justice Department over its dominant market share in AI chips and sales practices. The company is rapidly building its legal and policy teams to address antitrust concerns amid profitable growth, as it commands 90 percent of the GPU market essential for AI systems. Nvidia is also adapting to increased competition oversight, with recent attention turning to its planned acquisition of Run.ai and impact on the AI supply chain.

Intel has announced its new Gaudi 3 AI processors, claiming up to 1.7X the training performance, 50% better inference, and 40% better efficiency than Nvidia's H100 processors at a lower cost.

Intel has announced its new Gaudi 3 AI processors, claiming up to 1.7X the training performance, 50% better inference, and 40% better efficiency than Nvidia's H100 processors at a lower cost.

Meta has announced the next generation of its AI accelerator chip. Its development focused on chip memory (128GB at 5nm) and throughput (11 TFLOPs at int8).

Nvidia became the second most valuable company in the world on Wednesday afternoon as its market capitalization hit $3.01 trillion. It became a $1 trillion company in May 2023, hitting $2 trillion in February this year. The company reported $14 billion in profit in May. Its AI accelerators make up between 70% and 95% of the market share for AI chips. Nvidia has plans to launch a new AI chip every year.

AMD unveiled its latest AI processors, including the MI325X accelerator due in Q4 2024, at the Computex trade show. It also detailed plans to compete with Nvidia by releasing new AI chips annually. The MI350 series, expected in 2025, promises a 35-fold performance increase in inference compared to the MI300 series. The MI400 series is set for a 2026 release.

Intel Foundry has completed assembly of the first commercial High-NA EUV lithography machine, an advancement critical for its 14A process node R&D targeted for 2025. The High-NA tool, by ASML, enables smaller feature printing and up to 2.9X transistor density improvement. Intel's early adoption of this technology aims to overcome past delays and regain leadership in semiconductor process technology.

Etched has announced a $120M round led by Primary Venture Partners and Positive Sum Ventures, as well as angel investors including Peter Thiel, Stanley Druckenmiller, and David Siegel. It is working directly with TSMC's Emerging Business Group to produce chips on their state-of-the-art, 4 nanometer node. Its chips can achieve 500K tokens/second. One 8xSohu can replace 160 H100s.

Bitdeer Technologies, a Nasdaq-listed bitcoin mining firm based in Singapore, has successfully tested its first self-designed 4-nanometer process technology crypto mining chip, the SEAL01. This advanced chip is designed to optimize bitcoin mining performance while reducing power consumption, leading to lower operating costs and a potentially smaller environmental footprint.

Google has recently introduced AlphaChip, a groundbreaking AI-assisted chip design technology that utilizes reinforcement learning to optimize chip layouts. This innovative approach significantly accelerates the design process, allowing for the creation of chip floorplans in just a few hours, compared to the traditional timeline of up to 24 months for complex chips. The technology aims to enhance performance, power efficiency, and overall design quality, making it a valuable tool for companies like Google and MediaTek, which have already begun implementing it in their chip designs. Historically, chip design has been a labor-intensive and costly endeavor, particularly during the floorplanning phase. While existing AI-assisted tools have emerged, they often come with high costs, limiting accessibility. Google’s AlphaChip seeks to democratize this technology, making it more available to a broader range of developers. The system operates by treating chip floorplanning as a game, where it places circuit components on a grid and learns from each layout it creates, improving its efficiency over time. Since its inception in 2020, AlphaChip has been instrumental in designing Google's Tensor Processing Units (TPUs), which are crucial for powering various AI models and cloud services. The technology has evolved with each generation of TPUs, including the latest 6th Generation Trillium chips, enhancing their performance and reducing development time. Although AlphaChip has shown remarkable capabilities, human developers still play a significant role in the design process, particularly for more complex tasks. The success of AlphaChip has sparked interest in further research into AI applications across different stages of chip design, including logic synthesis and timing optimization. Google envisions a future where AI-driven optimization could revolutionize the entire chip design lifecycle, leading to faster, smaller, and more energy-efficient chips. As AlphaChip continues to develop, its applications may expand beyond current uses, potentially impacting a wide range of technologies in the future. In summary, Google’s AlphaChip represents a significant advancement in chip design technology, leveraging AI to streamline processes and improve outcomes. Its ongoing development and application could reshape the semiconductor industry, making chip design more efficient and accessible.

AlphaChip has significantly transformed the landscape of computer chip design through the application of advanced AI techniques. Initially introduced in a preprint in 2020, AlphaChip employs a novel reinforcement learning method to optimize chip layouts, which has since been published in Nature and made available as open-source software. This innovative approach has enabled the creation of superhuman chip layouts that are now integral to hardware utilized globally. The development of AlphaChip was motivated by the need to enhance the efficiency of chip design, a process that has historically been labor-intensive and time-consuming. Traditional methods could take weeks or months to produce a chip layout, whereas AlphaChip can generate comparable or superior designs in just hours. This acceleration is particularly evident in the design of Google’s Tensor Processing Units (TPUs), which are crucial for scaling AI models based on Google's Transformer architecture. AlphaChip operates by treating chip floorplanning as a game, akin to how AlphaGo and AlphaZero approached their respective games. It begins with a blank grid and strategically places circuit components, receiving rewards based on the quality of the final layout. A unique edge-based graph neural network allows AlphaChip to learn the intricate relationships between interconnected components, improving its performance with each design iteration. The impact of AlphaChip extends beyond Google’s internal projects; it has influenced the broader chip design industry. Companies like MediaTek have adopted and adapted AlphaChip to enhance their own chip development processes, leading to improvements in power efficiency and performance. The technology has sparked a wave of research into AI applications for various stages of chip design, including logic synthesis and macro selection. Looking ahead, the potential of AlphaChip is vast. It is expected to optimize every phase of the chip design cycle, from architecture to manufacturing, thereby revolutionizing the creation of custom hardware found in everyday devices. Future iterations of AlphaChip are in development, with the aim of producing chips that are faster, cheaper, and more power-efficient, ultimately benefiting a wide range of applications from smartphones to medical devices. The collaborative efforts of a diverse team of researchers have been instrumental in the success of AlphaChip, highlighting the importance of interdisciplinary work in advancing technology. As the field of AI-driven chip design continues to evolve, AlphaChip stands at the forefront, promising to reshape the future of computing.

Digital computing is reaching relative maturity, with traditional electronics well into the final phase of their S-curve and GPUs in the middle of theirs. The demand for compute is showing no sign of slowing. Next-gen approaches will need to harness entirely different materials systems with more amenable physics. This article looks at some of the possible technologies that may bring the industry to the next level.

Intel has unveiled its Core Ultra 200V lineup, previously known as Lunar Lake, boasting superior AI performance, fast CPUs, and competitive integrated GPUs for thin laptops. The processors feature eight CPU cores, integrated memory, and enhanced efficiency but are limited to 32GB RAM. Major manufacturers like Acer, Asus, Dell, and HP will launch laptops with these new chips. Reviews are pending to confirm Intel's claims.

VC firm Andreessen Horowitz has secured thousands of AI chips, including Nvidia H100 GPUs, to dole out to its AI portfolio companies in exchange for equity.

The US Department of Justice has sent subpoenas to Nvidia and other companies seeking evidence that the chipmaker violated antitrust laws. Antitrust officials are concerned that Nvidia is making it harder to switch to other suppliers and penalizing buyers that don't exclusively use its artificial intelligence chips. Nvidia claims that its market dominance stems from the quality of its products. The company prioritizes customers who can make use of its products in ready-to-go data centers as soon as they're provided to prevent stockpiling and to speed up the broader adoption of AI.

AlphaChip has significantly transformed the landscape of computer chip design through the application of advanced AI techniques. Initially introduced in a preprint in 2020, AlphaChip employs a novel reinforcement learning method to optimize chip layouts, which has since been published in Nature and made available as open-source software. This innovative approach has enabled the creation of superhuman chip layouts that are now integral to hardware utilized globally. The development of AlphaChip was motivated by the complexities inherent in chip design, which involves numerous interconnected components and intricate design constraints. For over sixty years, automating the chip floorplanning process has posed a challenge for engineers. AlphaChip addresses this by treating chip layout design as a game, akin to how AlphaGo and AlphaZero approached board games. It begins with a blank grid and strategically places circuit components, receiving rewards based on the quality of the final layout. This process is enhanced by an edge-based graph neural network that helps AlphaChip learn the relationships between components, allowing it to improve with each design iteration. Since its inception, AlphaChip has been instrumental in designing layouts for Google’s Tensor Processing Units (TPUs), which are crucial for scaling AI models based on Google’s Transformer architecture. These AI accelerators are foundational to various Google services and are also available to external users through Google Cloud. The pre-training phase of AlphaChip involves practicing on diverse chip blocks from previous TPU generations, which enables it to generate high-quality layouts for current designs. As a result, AlphaChip has consistently produced better layouts with each new TPU generation, significantly accelerating the design cycle and enhancing chip performance. The impact of AlphaChip extends beyond Google, influencing the broader chip design industry and research community. Companies like MediaTek have adopted and adapted AlphaChip to enhance their own chip development processes, demonstrating its versatility and effectiveness. The success of AlphaChip has sparked a surge of interest in applying AI to various stages of chip design, including logic synthesis and macro selection. Looking ahead, the potential of AlphaChip is vast, with aspirations to optimize every aspect of the chip design cycle, from architecture to manufacturing. Future iterations of AlphaChip are in development, with the goal of further revolutionizing chip design for a wide array of applications, including smartphones, medical devices, and agricultural sensors. The ongoing collaboration with the research community aims to create chips that are faster, more cost-effective, and energy-efficient, paving the way for the next generation of technology.

Cerebras, a California-based company, has demonstrated that its second-generation wafer-scale engine is significantly faster than the world's faster supercomputer in molecular dynamics calculations. It can also perform sparse large language model inference at one-third of the energy cost of a full model without losing any accuracy. Both achievements are possible due to the interconnects and fast memory access enabled by Cerebras' hardware. Cerebras is looking to extend the applications of its wafer-scale engine to a larger class of problems, including molecular dynamics simulations of biological processes and simulations of airflow around vehicles.

Cerebras, a California-based company, has demonstrated that its second-generation wafer-scale engine is significantly faster than the world's faster supercomputer in molecular dynamics calculations. It can also perform sparse large language model inference at one-third of the energy cost of a full model without losing any accuracy. Both achievements are possible due to the interconnects and fast memory access enabled by Cerebras' hardware. Cerebras is looking to extend the applications of its wafer-scale engine to a larger class of problems, including molecular dynamics simulations of biological processes and simulations of airflow around vehicles.

Intel gave the first architectural details of its Gaudi 3 third-generation AI processor at Vision 2024 this week in Phoenix, Arizona. Gaudi 3 is made up of two identical silicon dies, each with a central region of 48 megabytes of cache memory, joined by a high-bandwidth connection, surrounded by four engines for matrix multiplication and 32 programmable units called tensor processor cores. It produces double the AI compute of Gaudi 2 using 8-bit floating-point infrastructure. It also provides a fourfold boost for computations using the BFloat 16 number format.

Intel gave the first architectural details of its Gaudi 3 third-generation AI processor at Vision 2024 this week in Phoenix, Arizona. Gaudi 3 is made up of two identical silicon dies, each with a central region of 48 megabytes of cache memory, joined by a high-bandwidth connection, surrounded by four engines for matrix multiplication and 32 programmable units called tensor processor cores. It produces double the AI compute of Gaudi 2 using 8-bit floating-point infrastructure. It also provides a fourfold boost for computations using the BFloat 16 number format.